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CLAIMS 

1. An image recognition system that recognizes motions of 
players of a sport playing between areas partitioned by an 
obstacle such as a net from a content of its on-air program^ 
5 of its material video before broadcasted, or recorded by a 
recording media such as a VTR, and characterized by 
comprising 

a visual information obtaining part that obtains visual 
information on which a motion of the player in at least one 

10 area during a match is shown from the content, 

an occlusion state determining part that determines whether 
or not a used material such as a ball that moves between the 
areas included in the visual information obtained by the 
visual information obtaining part and that is an object to 

15 count the score of the relevant sport is in a state being 
hidden by a predetermined object body, 

an irtpact time information specifying part that specifies an 
impact time when the used material is hit based on an 
occlusion start time when the occlusion state determining 

20 part determines that the used material changes its state 

from not being hidden by the object body to being hidden by 
the object and an occlusion release time when the occlusion 
state determining part determines that the used material 
changes its state from being hidden by the object body to 

25 not being hidden by the object, 

a rule information storing pcurt that stores rule information 
to conduct the relevant sport, and 

an image content recognizing part that recognizes an image 
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content including the motion of the player shown by the 
visual information based on the visual information obtained 
by the visual information obtaining part, a position of the 
used material at the impact time specified by the impact 
5 time information specifying part and the rule information 
stored in the rule information storing part. 

2. The image recognition system described in claim 1, and 
characterized by that the occlusion state determining part 

10 comprises a distance determining part that determines 
whether or not the used material locates within a 
predetermined distance from the object body and an occlusion 
start and release time specifying part that specifies a 
moment when the distance determining part determines that 

15 the used material locates within the predetermined distance 
from the object. body and the used material changes its state 
from not being hidden by the object body to being hidden by 
the object body as the occlusion start time and that 
specifies a moment when the distance determining part 

20 determines that the used material locates within the 

predetermined distance from the object body and the used 
material changes its state from being hidden by the object 
body to not being hidden by the object body as the occlusion 
release time. 



3. The image recognition system described in claim 1 or 2, 
and characterized by that the impact time shown by t^f the 
occlusion start time shown by tO and the occlusion release 
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time shown by tl have a relationship shown by the following 
expression (Expression 1) 
(Expression 1) t^^aX tO+ (1-a) Xtl 
where coefficient a is O^a^l 

5 

4. The image recognition system described in claim 1^ 2 or 3^ 
and characterized by that the visual information obtaining 
part coitiprises a domain element extracting part that 
extracts used facilities information such as the obstacle 

10 like the net or a boundary line showing a boundary between 
the areas and outside the areas, player position information 
showing a position of the player and used material 
information showing the used material that moves between the 
areas and that becomes the object to count the score of the 

15 relevant sport from the visual information. 

5. The image recognition system described in claim 4, and 
characterized by that the player position information is 
position information showing a region including the player 

20 and a tool that the player always holds and uses during the 
match. 

6. The image recognition system described in claim 4 or 5, 
and characterized by that the domain element extracting part 

25 extracts the player position information from the visual 
information based on the used facilities information 
extracted by the domain element extracting part. 
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7. The image recognition system described in claim A, 5 or 6^ 
and characterized by that the domain element extracting part 
extracts the used material information from the visual 
information based on the used facilities information and the 

5 player position information extracted by the domain element 
extracting part. 

8. The image recognition system described in claim 4, 5, 6 
or 7, and characterized by that the used facilities 

10 information, the player position information, the used 
. material information and the rule information are based on 
knowledge regarding a sport item to be a target to extract 
the image. 

15 9. The image recognition system described in claim 1, 2, 3, 
4, 5, 6, 7 or 8, and characterized by cortprising an audio 
information obtaining part that obtains audio information 
synchronous with the visual information, such as an impact 
sound generating at a moment when the used material such as 

20 the ball that moves between the areas and that is an object 
to count the score of the relevant sport is hit from the 
content, wherein the impact time information specifying part 
specifies the impact time based on a combination of the 
occlusion start time and the occlusion release time and the 

25 audio information obtained by the audio information 
obtaining part. 

10. The image recognition system described in claim 9, and 
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characterized by that the impact time information specifying 
part specifies a time when the audio information shows a 
value bigger than a predetermined level as the impact time. 

5 11. The image recognition system described in claim 9 or 10, 
and characterized by that the audio information obtaining 
part comprises a filtering part that passes a predetermined 
frequency band and the audio information is the information 
that has passed through the filtering part. 

10 

12. The image recognition system described in claim 11, and 
characterized by the filtering part consists of a band-pass 
filter. 

15 13. The image recognition system described in claim 9, 10, 
11 or 12, and characterized by that the impact time 
information specifying part specifies the impact time based 
on an iirpact sound candidate data having a predetermined 
time including the impact sound extracted from the audio 

2 0 information . 

14. The image recognition system described in claim 9, 10, 
11 or 12, and characterized by that multiple pieces of 
impact sound candidate data are extracted from the audio 
25 information so that an inpact sound candidate data at one 
time and an impact sound ceuididate data at its subsequent 
time have a time that overlaps each other and the impact 
time information specifying part specifies the impact time 
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based on the multiple pieces of the impact sound candidate 
data. 

15. The image recognition system described in claim 14, and 
5 characterized by that each of the multiple pieces of the 

impact sound candidate data is arranged to have an identical 
data length and the multiple pieces of the impact sound 
candidate data are arranged to be extracted from the audio 
information at intervals of a certain period. 

10 

16. The image recognition system described in claim 9, 10, 
11, 12, 13, 14 or 15, and characterized by comprising an 
impact sound pattern information storing part that stores 
impact sound pattern information that is a patternized sound 

15 change due to a state under which the used material is hit 
by the tool such as a racket that the player always holds 
and uses during the match, wherein the iitpact time 
information specifying part specifies the impact time based 
on the impact sound pattern information stored in the impact 

20 sound pattern infomiation storing part and the audio 
information . 

17. An image recognition system that recognizes motions of 
players of a sport playing between eoreas partitioned by an 

25 obstacle such as a net from a content of its on-air program, 
of its material video before broadcasted, or recorded by a 
recording media such as a VTR, and characterized by 
comprising 
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a visual information obtaining part that obtains visual 
information on which a motion of the player in at least one 
area during a match is shown from the content, 
an occlusion state determining part that determines whether 
5 or not a used material such as a ball that moves between the 
areas included in the visual information obtained by the 
visual information obtaining part and that is an object to 
count the score of the relevant sport is in a state being 
hidden by a predetermined object body, 

10 an impact time information specifying part that specifies an 
impact time when the used material is hit based on an 
occlusion start time when the occlusion state determining 
part determines that the used material changes its state 
from not being hidden by the object body to being hidden by 

15 the object and an occlusion release time when the occlusion 
state determining part determines that the used material 
changes its state from being hidden by the object body to 
not being hidden by the object, and 

an image content recognizing part that recognizes an image 
20 content including the motion of the player shown by the 

visual information based on the visual information obtained 
by the visual information obtaining part and a position of 
the used material at the impact time specified by the intact 
time information specifying part. 



18. The image recognition system described in claim 17, and 
characterized by comprising an audio information obtaining 
part that obtains audio information synchronous with the 
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visual information, such as an impact sound generating at a 
time when the used material such as the ball that moves 
between the areas and that is an object to count the score 
of the relevant sport is hit from the content, and the 
5 impact time information specifying part specifies the impact 
time based on a combination of the occlusion start time and 
the occlusion release time and the audio information 
obtained by the audio information obtaining part, 

10 19. An image recognition program that is a program 

activating an image recognition system that recognizes 
motions of players of a sport playing between areas 
partitioned by an obstacle such as a net from a content of 
its on-air program, of its material video before broadcasted, 

15. or recorded by a recording media such as a VTR by operating 
a conputer, and characterized by making the computer 
function as 

a visual information obtaining means that obtains visual 
information on which a motion of the player in at least one 

20 area during a match is shown from the content, 

an occlusion state determining means that determines whether 
or not a used material such as a ball that moves between the 
areas included in the visual information obtained by the 
visual information obtaining means and that is an object to 

25 count the score of the relevant sport is in a state being 
hidden by a predetermined object body, 

an inpact time information specifying means that specifies 
an impact time when the used material is hit based on an 
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occlusion start time when the occlusion state determining 
means determines that the used material changes its state 
from not being hidden by the object body to being hidden by 
the object and an occlusion release time when the occlusion 
5 state determining means determines that the used material 
changes its state from being hidden by the object body to 
not being hidden by the object^ 

a rule information storing means that stores rule 
information to conduct the relevant sport, and 

10 an image content recognizing means that recognizes an image 
content including the motion of the player shown by the 
visual information based on the visual information obtained 
by the visual information obtaining means, a position of the 
used material at the impact time specified by the impact 

15 time information specifying means and the rule information 
stored in the rule information storing means. 

20. An image recognition program that is a program 
activating an image recognition system that recognizes 

20 motions of players of a sport playing between areas 

partitioned by an obstacle such as a net from content of its 
on-air program, of its material video before broadcasted, or 
recorded by a recording media such as a VTR by operating a 
computer, and characterized by making the computer to 

25 function as 

a visual information obtaining means that obtains visual 
information on which a motion of the player in at least one 
area during a match is shown from the content. 



47 



an occlusion state detentdning means that determines whether 
or not a used material such as a ball that moves between the 
areas included in the visual information obtained by the 
visual information obtaining means and that is an object to 
5 count the score of the relevant sport is in a state being 
hidden by a predetermined object body, 

an impact time information specifying means that specifies 
an iitpact time when the used material is hit based on an 
occlusion start time when the occlusion state determining 

10 means determines that the used material changes its state 

from not being hidden by the object body to being hidden by 
the object and an occlusion release time when the occlusion 
state determining means determines that the used material 
changes its state from being hidden by the object body to 

15 not being hidden by the object, and 

an image content recognizing means that recognizes an image 
content including the motion of the player shown by the . 
visual information based on the visual information obtained 
by the visual information obtaining means and a position of 

20 the used material at the impact time specified by the impact 
time information specifying means. 



